Grounding Artificial Intelligence in the Origins of Human Behavior

Recent advances in Artificial Intelligence (AI) have revived the quest for agents able to acquire an open-ended repertoire of skills. However, although this ability is fundamentally related to the characteristics of human intelligence, research in this field rarely considers the processes that may have guided the emergence of complex cognitive capacities during the evolution of the species. Research in Human Behavioral Ecology (HBE) seeks to understand how the behaviors characterizing human nature can be conceived as adaptive responses to major changes in the structure of our ecological niche. In this paper, we propose a framework highlighting the role of environmental complexity in open-ended skill acquisition, grounded in major hypotheses from HBE and recent contributions in Reinforcement learning (RL). We use this framework to highlight fundamental links between the two disciplines, as well as to identify feedback loops that bootstrap ecological complexity and create promising research directions for AI researchers

The Ecology of Open-Ended Skill Acquisition: Computational framework and experiments on the interactions between environmental, adaptive, multi-agent and cultural dynamics

An intriguing feature of the human species is our ability to continuously invent new problems and to proactively acquiring new skills in order to solve them: what is called open-ended skill acquisition (OESA). Understanding the mechanisms underlying OESA is an important scientific challenge in both cognitive science (e.g. by studying infant cognitive development) and in artificial intelligence (aiming at computational architectures capable of open-ended learning). Both fields, however, mostly focus on cognitive and social mechanisms at the scale of an individual’s life. It is rarely acknowledged that OESA, an ability that is fundamentally related to the characteristics of human intelligence, has been necessarily shaped by ecological, evolutionary and cultural mechanisms interacting at multiple spatiotemporal scales. In this thesis, I present a research program aiming at understanding, modelingand simulating the dynamics of OESA in artificial systems, grounded in theories studying its eco-evolutionary bases in the human species. It relies on a conceptual framework expressing the complex interactions between environmental, adaptive, multi-agent and cultural dynamics. Three main research questions are developed and I present a selection of my contributions for each of them.- What are the ecological conditions favoring the evolution of skill acquisition?- How to bootstrap the formation of a cultural repertoire in populations of adaptive agents?- What is the role of cultural evolution in the open-ended dynamics of human skill acquisition?By developing these topics, we will reveal interesting relationships between theories in human evolution and recent approaches in artificial intelligence. This will lead to the proposition of a humanist perspective on AI: using it as a family of computational tools that can help us to explore and study the mechanisms driving open-ended skill acquisition in both artificial and biological systems, as a way to better understand the dynamics of our own species within its whole ecological context. This document presents an overview of my scientific trajectory since the start of my PhD thesis in 2007, the detail of my current research program, a selection of my contributions as well as perspectives for future work

Multi-Agent Reinforcement Learning as a Computational Tool for Language Evolution Research: Historical Context and Future Challenges

Computational models of emergent communication in agent populations are currently gaining interest in the machine learning community due to recent advances in Multi-Agent Reinforcement Learning (MARL). Current contributions are however still relatively disconnected from the earlier theoretical and computational literature aiming at understanding how language might have emerged from a prelinguistic substance. The goal of this paper is to position recent MARL contributions within the historical context of language evolution research, as well as to extract from this theoretical and computational background a few challenges for future research

Studying the joint role of partial observability and channel reliability in emergent communication

International audienceMulti-Agent Reinforcement Learning (MARL) provides a powerful conceptual and computational framework for modeling emergent communication as a way to solve complex problems in sequential environments. However, despite the recent advances in this field, there is still a need to better understand the role of heterogeneous factors, e.g. partial observability and channel reliability, in the emergence of communication systems. An important step has recently been done in this direction by proposing new information-theoretic measures of emergent communication. As of yet, very few contributions have taken advantage of these new measures to perform detailed quantitative studies analyzing how different environmental and cognitive factors can foster the emergence of communication systems. This work quantitatively measures the joint role of partial observability and channel reliability in the emergence of communication systems. To this end, we performed experiments in a simulated multi-agent grid-world environment where agents learn how to solve different cooperative tasks through MARL

Learning how to reach various goals by autonomous interaction with the environment: unification and comparison of exploration strategies

International audienceIn the field of developmental robotics, we are particularly interested in the exploration strategies which can drive an agent to learn how to reach a wide variety of goals. In this paper, we unify and compare such strategies, recently shown to be efficient to learn complex non-linear redundant sensorimotor mappings. They combine two main principles. The first one concerns the space in which the learning agent chooses points to explore (motor space vs. goal space). Previous works have shown that learning redundant inverse models could be achieved more efficiently if exploration was driven by goal babbling, triggering reaching, rather than direct motor babbling. Goal babbling is especially efficient to learn highly redundant mappings (e.g the inverse kinematics of a arm). At each time step, the agent chooses a goal in a goal space (e.g uniformly), uses the current knowledge of an inverse model to infer a motor command to reach that goal, observes the corresponding consequence and updates its inverse model according to this new experience. This exploration strategy allows the agent to cover the goal space more efficiently, avoiding to waste time in redundant parts of the sensorimotor space (e.g executing many motor commands that actually reach the same goal). The second principle comes from the field of active learning, where exploration strategies are conceived as an optimization process. Samples in the input space (i.e motor space) are collected in order to minimize a given property of the learning process, e.g the uncertainty or the prediction error of the model. This allows the agent to focus on parts of the sensorimotor space in which exploration is supposed to improve the quality of the model

Curiosity-driven phonetic learning

International audienceThis article studies how developmental phonetic learning can be guided by pure curiosity-driven exploration, also called intrinsically motivated exploration. Phonetic learning refers here to learning how to control a vocal tract to reach acoustic goals. We compare three different exploration strategies for learning the auditory-motor inverse model: random motor exploration, random goal selection with reaching, and curiosity-driven active goal selection with reaching. Using a realistic vocal tract model, we show how intrinsically motivated learning driven by competence progress can generate automatically developmental structure in both articulatory and auditory modalities, displaying patterns in line with some experimental data from infants

Exploration strategies in developmental robotics: a unified probabilistic framework

International audienceWe present a probabilistic framework unifying two important families of exploration mechanisms recently shown to be efficient to learn complex non-linear redundant sensorimotor mappings. These two explorations mechanisms are: 1) goal babbling, 2) active learning driven by the maximization of empirically measured learning progress. We show how this generic framework allows to model several recent algorithmic architectures for exploration. Then, we propose a particular implementation using Gaussian Mixture Models, which at the same time provides an original empirical measure of the competence progress. Finally, we perform computer simulations on two simulated setups: the control of the end effector of a 7-DoF arm and the control of the formants produced by an articulatory synthesizer

The role of intrinsic motivations in learning sensorimotor vocal mappings: a developmental robotics study

International audienceLearning complex mappings between various modalities (typically articulatory, somato­sensory and auditory) is a central issue in computationally modeling speech acquisition. These mappings are generally non­linear and redundant, involving high dimensional sensorimotor spaces. Classical approaches consider two separate phases: a relatively pre-determined exploration phase analogous to infant babbling followed by an exploitation phase involving higher level communicative motivations. In this paper, we consider the problem as a developmental robotics one, in which an agent actively learns sensorimotor mappings of an articulatory vocal model. More specifically, we show how intrinsic motivations can allow the emergence of efficient exploration strategies, driving the way a learning agent will interact with its environment to collect an adequate learning set

COSMO (“Communicating about Objects using Sensory–Motor Operations”): A Bayesian modeling framework for studying speech communication and the emergence of phonological systems

International audienceWhile the origin of language remains a somewhat mysterious process, understanding how human language takes specific forms appears to be accessible by the experimental method. Languages, despite their wide variety, display obvious regularities. In this paper, we attempt to derive some properties of phonological systems (the sound systems for human languages) from speech communication principles. We introduce a model of the cognitive architecture of a communicating agent, called COSMO (for “Communicating about Objects using Sensory–Motor Operations') that allows a probabilistic expression of the main theoretical trends found in the speech production and perception literature. This enables a computational comparison of these theoretical trends, which helps us to identify the conditions that favor the emergence of linguistic codes. We present realistic simulations of phonological system emergence showing that COSMO is able to predict the main regularities in vowel, stop consonant and syllable systems in human languages

A Unified Theoretical Bayesian Model of Speech Communication

International audienceBased on a review of models and theories in speech communication, this paper proposes an original Bayesian framework able to express each of them in a unified way. This framework allows to selectively incorporate motor processes in perception or auditory representations in production, thus implementing components of a perceptuo-motor link in speech communication processes. This provides a basis for future computational works on the joint study of perception, production and their coupling in speech communication